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(54) NONLINEAR EDITING DEVICE AND NONLINEAR EDITING METHOD 



(57) Hard disk un'rt (101) stores a plurality of 
encoded video data That Is encoded in a plurality at 
compression formats, and the encoded video data Is 
arbitrarily read out. The encoded video data read from 
hard disk unit (101) Is decoded by first multi-format 
decoder (1 02) or second multi-format decoder (1 04) in 
response to a command of editing control section (1 09) ( 
and Is supplied to first format converter (103) or second 
format converter (105). The data that is converted in 
relation to video format by each format converter is syn- 
thesized by DVE (106), is converted in relation to video 
format by third format converter (107), and js outputted 
from video output section (108). 
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D**orTpt1on 

FIELD OF THG INVENTION 

(OOOtl "T~h e pra«ant ir-t-v** ntlon nalates to a devloo for 

editing video data onooded I r> dirrerent comproojalon for- 
mate and I to rr-ie>tr-»od. 



In 



rvi f 



BAOKQROUND Of" THE 



INVENTION 



rooo rda rt an rnagn^tlo taps 604. 

tOO OOJ PEG dacod*r 60S roproducM tho vldao 

data recorded on magnetic tap* QO-«» »nd docomp re^seec 
rt to bono D«nd d atd. 

[aooD] MPEQ deoodor 607 reproduces video 

aouroa dntn OOO recorded on magn«tto lope 606 e»nd 
decomprene* vldoo source data ©OS to base band 



[0002] Conventionally, vlooo for toieoastlng pro- 

grams or the ItKe has been produced by edlilna -video 
oourcio (Into recorded on a mognatio tnpo ualng a VCR 
<Wcloo OM»<»tte Recordar). Rcfcorrtfy, the video source 
data has been recorded an digital dmta. Th o video 
aounco data rooordad q> th« a Igttial data lea ancodoa I n 
compression formotn amployod by raopactlva recording 
devices. As tho oomproMio n formate, thoro ere aevoral 
klnda of formats auch 03 N/1 P" EfGi (Mowl rig Plctura 
Expert* Qroup> forrriBit,' a motlon-JPEO (Joint Photo- 
graphic Experts Group) to 1 « 1 » at., a. format (It Us hsreinarrter 
oollad C3V, aapaolajly la oa)l*d DVMD tor Wp) daoM«d try 
H O (Hlgh-Dof Initio n) Digital VCR ConTaranoe, and st. To r- 
mot (one hnvino 2B Mbpa of enooded video rata ic* horo- 
inoftor col led ON/OF* RO. end one having SO Ivlbps of 
encoded video rate le celled DVCPR050) daflned In 
SMPTG 3T-a.lv! or 3k/RTE (Society or rutetlon Picture end 
Television Engmeers) standRTtlv. A compreeelort format 
employed by eecn reoordlng device dopondo on the 
device, and an employed magnetic tapo vorl«M. 
[0003] Por resolvtng troublcsomon«3B that the 

vldoo source data recorded on vedaua magnetic tap as 
le managed, trie 00 mp> roc-do n format of tJ-ie video 
sou roe data is often oonverted gutO untried In an edttlng 
worK proeeee. end the video eouroa dete oorwerted in 
compression format is reoorded and otorod or* a mag- 
netic tape In the aeme format. In other words, trie vldao 
sourca aata recoroea in venous compression rormato la 

a magneao tape atraohabte to a. VCB for 
reproducing; ocfltlng ecu roes, end te nrchtvod. 
[0004] Fig. e ia a block diagram of a oortvorrtionQ] 

edltlno oytrtem . 

[OOOB] THI« editing «y«t«rn produces broadcasting 

video date in tho MPE<3 Tormat using vldoo sou roe do to 
reoorded in tn*» C^V format end v]doo source data 
rasa r-ded In the MPEQ format. 

lODOe) In Fib- B. vVdao source data Is rooordod on 

magnetic tape 601 In the OV forrnot. Another video 
•ourcn data le recorded on megnetlc tape oOO In tho 
Jv/I F» format. For convenience sake, the video source 
dete recorded on megnetlc tape 601 te oetled video 
oourco data 681 . ond tho video source dote recorded on 
magnetic tape 606 ie celled video source date 00O. 
[OOOTJ First. Drv decoder 002 reproduces video 

source darts, eo "1 recorded on rnagnatle tape SOI and 
docompraesec video souroe data oai to b>ojeo bond 
data, fvl F* E£GS enooder 603 00m p roes es the 



Digital vldao efTeertc 
> base band datB 
JOS and MPEQ dace 
autputo tho base bn 



r (OVE) •ynihe»te«« 
eupoliod from MPEQ 
der 607 with a given tim- 
id data give n a dleeolve 



fooii] MPEO deooder eo« oompreeeee 

bftnd data supplied from CA/E 603 in the rvlFE* 
end reoorde tt on mognatio tope eio. 
[0012] Hnolly the video source data Vo etorad on 

megnatlc tapss 604. cue, and tha edited video data Is 
etorad on magnetic taps 81 Q>- Magnotle topes aO^a. OOo. 
OlO nave the seme tormat, end the vldoo data as 
rooondtoc* In the eeme oomp roaoion format. 

tOOl 3J Tnuo. the convorrtionol edrtihO ay atom Uni- 

fies tho formate of the video souroe date recorded In 
various compression formats during In an editing work 
procsss. and thoreforo, a process whara tho vldao 
eouroe dam te rooomprooood after balng tomporarlly 
decompressed le required. "This process has a problem 
that original image quanry of trie video data ie notloeably 
degradBd. In addition, there Is a problem that a large 
spacs for storing tho magnetic tspa Is also raqulrad. 
[00-14] On the other nand, personal-computer- 

based nonlinear editing devlcas have recently become 
widespread. S\ general noniinearedlflng device digitize a 
e vtdeo eignal of a base bend, anoodea It in a prescribed 
compression format, stores video dote In a hard dak. 
and raanzaa space-saving, However, a procass whara 
tne video date recorded on tne magnetic tape is ropro- 
duoad by a VCR, la dooompreeeed to the beae band, 
and then Is digitized Is stJll required, and the Image deg- 
radation caused try the decompression or the reproduc- 
tion of the video data ie a problem. 

[OO-f 5] Gome type of nonlinear editing device can 

directly take In tho oompraessd vtdao data. In this case, 
aleo, the nonllnoor editing devloa cannot handle proo- 
eee a plurality of oompreeelon formate beoause It te lim- 
ited te e pra*orib«d oornprassion formnt . 

[OOl a] Thus. In the general nonllnoor editing device 

the compression format Is fixed, and no general nonlin- 
ear editing davlca can edit mlx»d-vldeo-eou roe-data In a 
plurality of compreeelon formats. 

□ IBCLOBURE OF THE INVENTION 

rpd Tl The present Invention provides a nonlinear 

adltlng dovrloa and a nonllnoar editing method whloh aan 
BVrninato a troubka«orno preevve Tor untrorrnlng formate 
and allow a hlgh-q utility edttlno work where degradation 



or vldao data following rocompreosi orn 
even when mixed video source data In 
compression formats oxlats. 

[OOl O] For addressing th a problems dlscuonad 

above, the nonlinear editing device In accordance with 
the present Invention comprises the following olerrtsnte: 

a etorege the* can rocoro vldoo date oompreeaed 
and onoooac I r» a plurality of Kinds of compression 
formalo Keeping the compression formats; end 
a first multl -format decoder that can docomproco 
tho vldao data racordod 1 r\ tha storage In at leaet 
two or more oomprseelon formate of the plurality of 
Klnde of 00m preset on formate. 

[OOl B] The nonlinear editing devtos in eoo-ordsnce 

with the present Invention furthsr Includes e second 
multi-Tormat do cod or that can decompress the vidao 
dote raeordod In tha storage In at least one or mora 
compression formats of tha plurality of Kinds of com- 
pression formats. 

[0020] "The nonlinear editing dovloo in accordance 

with tha present invention furth«r Incluaoo e digital video 
effractor for synthssizmg output data of the first multi-for- 
mat decoder and output data of tho aeoond mu Itl-format 



auditing device In 



ordanoe 



[OOZT] The nonllnei 

with tha present Inrventlo 
verter ror psrrermlng at laeet ona of &O (Standard Defi- 
nition) / HD (Might Definition) conversion. HD/'SO 
convomlon. IMTSC (National Television System Commit" 
too) / FAL (Phaee Alternation Lines) converalon. and 
PAuNTSC conversion. 

[00»J For aadrooclng the " proDlome dlsfiuseed 

above, the nonlinear eartlng method in accord a no a with 
the preeont I nvontlon comprlaee a flrat decoding stop for 
oaquontlalry deoompreee in g video data which is 
1 in the storage that can record the vldao date 
sod and encodad In the plurality of klndu of 
compro.««:lon rormau: Keeping the oompreeelon formats 
Bind ro in art leaet two or more compreselon formate of 
the plurality of Kinds of compression formete. 
COQ23] The nonllnoor editing method in accordance 

with the preaent Invention furthar Includes a second 
dooodlng step for see uentleJ ly dseompreeslng tr*e vldao 
date racorda-d In tho storage In at Isae* one or more 
compression formate of the plu rallty of klnda of com- 
preeslon form tats. 

[OQ24] The nonlinear editing method In accordance 

with the present Invention further Indudso a video affect 
step for synthesizing the output data obtained from trio 
first decoding ctop end tha output data obtained from 
the second decoding stop. 

[O02«l The nonlinear editing method in aooordenoe 

wttn the preaent invention further Includes a format con- 
verting step for performing art least one of the SO/H CZ> 
conversion, the HD/3D conversion, the NTSC/PAL 00 h- 
vsrslon, and the RALVNITSC converalon. 



BFtiER DE 
[0026] 

Rig. I 



tORIPTION OF THE DFtyVVl/IKIC3S 



[ a block dlegram of a nonllnt 
> occordence -with oxamplnry en 
1 of the present Invention. 

Fig. £ le an operation timing chart or tna 
editing device in eooordanoe wllh 
embodiment I of the present invention. 
Rig. 3 Is an operation timing chart of a 
editing device In accordance vtsttn 
embodiment of the present Invention. 
Fig. Is an operation timing chart of a 

editing device tn accordance 
embodiment 3 of the present 1 nventlc 
Pig. S la an operation timing chert of a nonlinear 
editing device In accordance wrth axomplory 
ombodlmont of the present Invention. 

Fig. a Is a DioaK diagram of o conventional nonlinear 
editing device. 



nonlinear 



PREFERREfJ EM BODIM ENT3 OF " 



rpoa7] Non llnoor editing devtoos In acOO rdanoo wTth 

four preferred ombodlme nte Of the present invention ere 
docorlbed hereinafter with reference to the drowlngs. 



(Preferred an 



ant t > 



[oo2b] Fig. 1 is a block diagram of a nonlinear edit- 

ing device I n accordance with embodiment 1 of the 
preeont Invention, in Fig. n . hard disk unit 1 01 stores a 
plurality of encoded video data that Is encoded In a plu- 
rality of compreselon formate, end the encoded video 
data is arbitrarily read out. Hard dlok unit 1 01 can 
receive ervcodeo video data that Is compreeeed and 
encoded from outside end Mors rt. In embodiment 1 . 
hard dtaK unit 101 otoree a plurality of enooded video 
date in each compression format of DVCPRO, 
DVCPROSO. DV, DVH o. ivip>eca. 

[OOSe] editing eontroi eaotlon 1 OO has e grephiool 

user mterfeoa <c=i_i|) (not shown In Flo. 1>. <*vn operator, 
using the <3 1_» I . oomblnse the plurality of encodod video 
data stored In hard diek unit 1 O -I end produces e desired 
video ooq i>a noe. The vldao cequanco te etored In an 
edit decision list of a reproduction starting point and e 
reproduction finishing point Of OO Oh encoded Video data, 
a roadlng-out order, video effect zone end type, and the 
I ike. Editing control section TOO controls tho entire non- 
linear edltlno device based on the edlrt decision list for 
reproducing the desired video eeeuence. 

too 30] The encoded video data read from hard dlSK 

unit -1 o 1 ia supplied to first mt_j itl-TormHt deooder 1 OS or 
second muitt-rormet decoder 1 O-+ in responoe to a com- 
mand of editing oontrol aeotlon -I OS. 

[00311 First multi-format decoder 1 02 can decode 

the enooded video date In the oompreeelon formate of at 
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least the DVCPRO and the DVCPRO50. Second multi- 
format decoder 1 04 can decode the encoded video data 
in the compression formats of at least the DVHD. 
[0032] Base band data decoded by first murti-for- 
mat decoder 1 02 Is sequentially fed Into first format con- 
verter 103. Base band data decoded by second mutti- 
format decoder 1 04 te sequentially fed into second for- 
mat converter 1 05. 

[0033] Rret and second format converters 103, 
105, in response to a command of editing control sec- 
tion 109, perform video format conversion such as 
NTSC/PAL conversion, PAL/NTS G conversion, SD/HD 
conversion, or HD/SD conversion, and output the con- 
verted base band data, in addition, first and second for- 
mat converters 103, 1 05 output the fed pa3e band data 
without any conversion in response, to a command of 
editing control section 1 09. 

[0034] Digital video effector (DVE) 106, in response 
to a command of editing control section 109, synthe- 
sizes base band data supplied from first and second for- 
mat converters 103, 105, applies a video effect such as 
a dissotve effect or a wipe effect to the data, and outputs 
it in addition, DVE 106 can output the base band data 
supplied from one of first and second format converters 
103, 105 without any conversion in response to a com- 
mand of editing control section 1 09- 
[0035] Third format converter 1 07, in response to a 
command of editing control section 109, performs a 
video format conversion such as the NTSCVPAL conver- 
sion, the PAL/NTSC conversion, the SD/HD conversion, 
or the HD/SD conversion, and outputs the base band 
data, in addition, third format converter 107 can output 
the fad base band data without any conversion in 
response to a command of editing control section 1 09. 
[0036] Video output section 10B supplies the base 
band data outputted from third format converter 107 to 
a monitor (not shown in Fig. 1). Analog output and dig- 
ital output are switched depending on a connected mon- 
itor. 

[0037] An operation flow of the nonlinear editing 
device having such structure in accordance with 
embodiment 1 is hereinafter described. 
[0033] Rg. 2 is an operation timing chart Of the non- 
linear editing device in accordance with exemplary 
embodiment 1 of the present invention- In Rg. 2, 
sequence 2100 is a control sequence based on an edit 
decision list stored in editing control section 109. 
Sequence 2200 shows a transition of a decoding oper- 
ation mode of first multi-format decoder 102. Sequence 
2400 shows a transition of a decoding operation mode 
of second mutti-format decoder 104. Sequence 2S00 
shows a transition of a converting operation mode of 
second format converter 105. Sequence 2600 shows a 
video output sequence of DVE 106. Sequence 2700 
shows a transition of a converting operation mode of 
third format converter 107- Sequence 2800 shows a 
transition of a video output operation of video output 
section 1 08. 



[0039] First first video data compressed and 
encoded in the DVCPRO format of a NTSC mode is 
read out from hard disk unit 101, and is supplied to first 
multi-format decoder 102. First multi-format decoder 
5 102 decodes the first video data in a DVCPRO format 
decompression mode. The base band data decoded by 
first multi-format decoder 102 is supplied to DVE 106 
through first format converter 1 03. 
[0040] Next, second video data compressed and 
10 encoded In the DV format of a HD mode Is read out from 
hard disk unit 101 . and is supplied to second multi-for- 
mat decoder 104. Second multi-format decoder 104 
decodes the second video data in a DVHD format 
decompression mode. HD base band data decoded by 
is second multi-format decoder 1 04 Is supplied to second 
format converter 105. Second format converter 105 
converts down the HD base band data decoded by sec- 
ond multi-format decoder 104 to SD base band data 
and supplies it to DVE 1 06. 
zo [0O41] Next, third video data compressed and 
encoded in the DVCPRO50 format of the NTSC mode is 
read out from hard disk unit 101. and Is supplied to first • 
mufti-format decoder 102. First multi-format decoder 
1 02 is switched from the DVCPRO format decompres- 
zs sion mode to a DVCPftOSO format decompression 
mode in lime for the timina and decodes the third video 
data. The base band data decoded by first multi-format 
decoder 1 02 Is supplied to DVE 1 06 through first format 
converter 103. 

30 [0042] DVE 106 applies the dissolve effect at a cut 
back point from a first video to a second video and the 
wipe effect at a cut back point from the second video to 
a third video, and outputs the video. Third format con- 
verter 1 07 converts the video output in the NTSC format 
35 of DVE 1 06 to the PAL format 

[0043] Video output section 10B outputs the video 
data convened toy third format converter 107 to the 
monltor- 

[0044] Thus, in embodiment 1 , even if compression 
40 formats are not unified, encoded video data keeping 
various compression formats can be stored in a hard 
disk un'rt 101 and is decoded adapted to respective 
compression formats. As a result, edited results are not 
required to be recorded on a magnetic tape to prevent 
45 recompression from causing degradation of image qual- 
ity. 

[0045] In embodiment 1 hard disk unit 101 is used 
as the storage, but, the storage is not limited to this. For 
example, other mass storage can be similarty used. 
so Compression formats are not limited to those In embod- 
iment 1 either. In addition, in embodiment 1 two multi- 
format decoders are provided, but it is not limiting. The 
format converter may be one capable of switching a plu- 
rality of format conversions with each other, and can be 
55 eliminated from the device structure. 
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(Preferred embodiment 2) 

[0046] Embodiment 2 is described hereinafter. A 
structure of a nonlinear editing device in accordance 
with embodiment 2 is same as that in embodiment 1 5 
where second multi-format decoder 1 04 is replaced with 
a software decoder. This software decoder is called 
third multi-format decoder 304. Third multi-format 
decoder 304 can decode encoded video data in at least 
MPEG format in a software process at a processing to 
period 1 .5 times tonger than a real time process period. 
[0047] Rg. 3 is an operation timing chart of the non- 
linear editing device in accordance with embodiment 2. 
In Fig. 3, sequence 3100 is a control sequence based 
on an edrt decision Dst stored in editing control section is 
1 09. Sequence 3200 shows a transition of a decoding 
operation mode of first muiti-format decoder 102. 
Sequence 3400 shows a transition of a decoding oper- 
ation mode of third multi-format decoder 304. Sequence 
3B00 shows a transition of a video output operation of go 
video output section iob. 

[0048] First, fourth video data compressed and 
encoded in the DrVCPRO format is read out from hard 
disk unit 101. and is supplied to first multi-format 
decoder 102. Rrst mufti-format decoder 102 decodes 25 
the fourth video data in a DVCPRO format decompres- 
sion mode. The data decoded by first multi-format 
decoder 102 is supplied to DVE 106 through first format 
converter 103. 

[00491 Next, fifth video data compressed and 30 
encoded in the MPEG format is read out from hard disk 
unit 101, and is supplied to third multi-format decoder 
304. Considering decoding process period of third 
multi-format decoder 304, the fifth video data is read out 
from hard disk unit 101 simultaneously with reading out 35 
of the fourth video data fiince the latter portion of the 
reading out of the fourth video data, and is supplied to 
third multi-format decoder 304, Third multi-format 
decoder 304 decodes the fffim video data tn a MPEG 
format decompression mode. The decoded base b^nd 4C 
data is buffered as appropriate, and is supplied to DVE 
106 through second format converter 1 05 with a suita- 
ble timing. 

£0050] Next, sixth video data compressed end 
encoded in the PVCPFIO50 format is read out from hard 45 
disk unit 101, and is supplied to first muiti-format 
decoder 102. Rrst mufti-format decoder 1 02 is switched 
from the DVCPRO format decompression mode to the 
DVCPRO50 format decompression mode in time for this 
timing, and decodes the sixth video data. The base st 
band data decoded by first multi-format decoder 102 is 
supplied to DVE 106 through first format converter 1 03. 
[0051] DV/E 106, based on control sequence 3100, 
performs timaiy cut back from the fourth video to the fifth 
video, and from the fifth video to the sixth video, and si 
outputs video data The video data outputted from DVE 
1 06 is supplied to video output section 1 08 through third 
format converter 1 07, and is supplied to a monitor. 



»9 A1 a 

[0052] Thus, in embodiment 2, the video data is 
previously read responsive to software decoding proc- 
ess period of third multi-format decoder 304, and a 
seamless video sequence is realized. 

(Preferred embodiment 3) 

[0053] Embodiment 3 is described hereinafter. A 
structure of a nonlinear editing device in accordance 
with embodiment 3 is same as that in embodiment 1 
where second multi-format decoder 1 04 is replaced with 
a software decoder. This software decoder i3 called • 
fourth multi-format decoder 404. Fourth mufti-format 
decoder 404 can decode encoded video data with size 
smaller than a prescribed value in the DVCPRO format 
and the OV format in real time in a software process. 
Furthermore, first muiti-format decoder 1 02 is consti- 
tuted so as to decode encoded video data in the MPEG 
format and the OV format as well as encoded video data 
in the PVCPRO format and the DVCPRO50 format. 
[00 54 J Fig. 4 is an operation timing chart of the non- 
linear editing device in accordance with embodiment 3. 
In Fig. 4, sequence 4100 is a control sequence based 
on an edit decision list stored in edtting control section 
109. Sequence 4200 shows a transition of a decoding 
operation mode of first mufti-format decoder 102. 
Sequence 4400 shows a transition of a decoding oper- 
ation mode of fourth mufti-format decoder 404. 
Sequence 4800 shows a transition of a video output 
operation of video output section 1 08. 
[0055] First, seventh video data compressed and 
encoded in the OVCP^Q format is read out from hard 
disk unit 101 ? and is supplied to first muiti-format 
decoder 102. First multMormat decoder 102 decodes 
the seventh video data in the DVCPRO format decom- 
pression mode. The base band data decoded by first 
multi-format decoder 102 is supplied to DVE 106 
through first format converter 103. Here, the latter por- 
tion of the seventh video data is decoded by fourth 
multi-format decoder 404. The base band data decoded 
by fourth multi-format decoder 404 is supplied to DVE 
106 through first format converter 103. During the latter 
portion of the seventh video data is decoded by fourth 
multi-format decoder 404, first muiti-format decoder 102 
is switched from the DVCPRO format decompression 
mode to the MPEG format decompression mode. 
[0056} Next, eighth video data compressed and 
encoded in the MPEG format is read out from hard disk 
unit 101, and is supplied to first multi-format decoder 
102. Before this timing, first multi-format decoder 102 
completes changeover of the decoding operation mode, 
and decodes the eighth video data m the MPEG format. 
The decoded base band data is supplied to DVE 10E 
through second format converter 105. 
r [0057] Next, the first portion of ninth video data 
compressed and encoded in the DV format is read out 
from hard disk unit 1 01, and is supplied to fourth muiti- 
format decoder 404. Fourth multi-format decoder 404 is 
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switched from the DVCPRO format decompression 
mode to the DV format decompression mode in time for 
this timing, and decodes the first portion of the ninth 
video data, me base band date decoded by fourth 
multi-format decoder 404 is supplied to DVE 106 
Through second format converter 105. 
[0056] During the first portion of the ninth video 
data Is decoded by fourth mufti-format decoder 404, first 
multi-format decoder 1 02 is switched from the MPEG 
format decompression mode to the DV format decom- 
pression mode. 

[0059] The latter portion of the ninth video data is 
supplied to first multi-format decoder 1 02. Before this 
timing, first multi-format decoder 102 completes 
changeover of the decoding operation mode, and 
decodes the latter portion of the ninth video data in the 
DV formal. The decoded base band data Is supplied to 
DYE 106 through first format converter 103- 
[0060] DYE 106, based on control sequence 4100, 
performs timely cut back from the first portion to the lat- 
ter portion of the seventh video, from the latter portion of 
the seventh video to the eighth video, from the eighth 
video to the first portion of the ninth video, and from the 
first portion to the latter portion of the ninth* video, and 
outputs video data, "me video data outputted from DVE 
1 06 is supplied to video output section 108 through third 
format converter 107, and is supplied to a monitor. 
[00611 Thus, in embodiment 3, a changeover period 
of decoding operation mode of first multi-format 
decoder 102 Is obtained by concurrently performing a 
software decoding process of fourth multi-format 
decoder 404, and a seamless video sequence is real- 
ized. 

(Preferred embodiment 4) 

[0062] Embodiment 4 is described hereinafter. A 
structure of a nonlinear editing device in accordance 
with embodiment 4 is same as that In embodiment 1 
where first multi-format decoder 102 is replaced with a 
decoder that can switch a decoding operation mode 
substantially seamlessly by means of a pipeline treat- 
ment technology or the like. Thla decoder is called sixth 
multi-format decoder 162. Second multi-format decoder 
104 is replaced with a software decoder, and this soft- 
ware decoder is called seventh multMormat decoder 
174. Seventh multMormat decoder 174, can decode 
encoded video data in at least the DV format in real time 
in a software process. 

[0063] Fig. 5 is an operation timing chart of the non- 
linear editing device in accordance with embodiment 4. 
In Ftg. 5, sequence 51QO is a control sequence based 
on an edit decision fist stored in editing control section 
1 09. Sequence 5200 shows a transition of a decoding 
operation mode of sixth multi-format decoder 162. 
Sequence 5400 shows a transition of a decoding oper- 
ation mode of seventh multi-format decoder 174. 
Sequence 5S00 shows a transition of a video output 
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operation of video output section 1 0B. 
[0064J First, tenth video data compressed and 
encoded in the MPEG format Is read out from hard disk 
unit 101, and is supplied to sixth multi-format decoder 
162. Sixth mum-format decoder 162 decodes the tenth 
video data in the DVCPRO format decompression 
mode. The base band data decoded by sixth multi-for- 
mat decoder 162 is supplied to DVE 106 through first 
format converter 1 03. 

[0065] The first portion of eleventh video data which 
is commanded with the dissolve effect Is decoded by 
seventh mufti-format decoder 174. The base band data 
decoded by seventh muW-format decoder 174 is sup- 
plied to DVE 106 through second format converter 1 05. 
[0066] The intermediate part of the eleventh video 
data which is compressed and encoded in the DV for- 
mat Is supplied, following the tenth video data, from 
hard disk unit 101 to sixth multi-format decoder 162. 
Sixth multi-format decoder 162 seamlessly completes 
changeover of the decoding operation mode, end 
decodes the intermediata part of the eleventh video 
data in the DV formal The decoded base band data is 
supplied to DVE 1 06 through first format converter 1 03. 
[0067] The latter portion (commanded with the wipe 
effect) of eleventh video data compressed and encoded 
in the DV format is supplied to seventh multi-format 
decoder 174. Seventh multi-format decoder 174 
decodes the fetter portion of the eleventh video data. 
The base band data decoded by seventh multi-format 
decoder 174 is supplied to DVE 106 through second for- 
mat converter 1 05. 

[0068] The twelfth video data compressed and 
encoded In the DVCPRO50 format ie supplied, following 
the intermediate part of the eleventh video data, from 
hard disk unit 101 to sixth multi-format decoder 162. 
Sixth multi-format decoder 162 seamlessly completes 
the changeover of the decoding operation mode, and 
decodes the twelfth video data in the DVCPRO format. 
The decoded base band data is supplied to DVE 106 
through first format converter 1 03, 

[0069] DVE 1 06 applies the video effect based on 
control sequence 5100 and outputs the video data. The 
video data outputted from DVE 106 is supplied to video 
output section 108 through third format converter 107, 
and Is supplied to a monitor. 

[0070] Thus, in embodiment 4, a seamless video 
sequence is realized by using, as a malh decoder, sbcth 
multi-format decoder 1 62 having a seamless changeo- 
ver function of the decoding operation mode, and sup- 
plementally operating seventh multi-format decoder 174 
only during a video effect period. 
[0071] In embodiments 1. 2. 3, 4 video data proc- 
esses are described. AudiD data accompanying the 
video data or independently inputted and stored audio 
data is processed similarly. The nonlinear editing device 
is not limited only to video editing. The nonlinear editing 
device can perform editing such as split, fade-in, fade- 
out and cross-fade with the video data for the audio 
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data. Furthermore, according to purposes, title data and 
Image data that are produced in various formats can be 
imported and overlapped on video. The video effect 
realteed by DVE 106 Is not limited to the wipe effect or 
the dissolve effect, and various video effects such as a 5 
picture-ln-plcture effect are available. Color collection 
can be applied to the base band data fed into DVE 1 06, 

INDUSTRIAL APPLICABILITY 

to 

[O072] The present Invention realises a nonlinear 
editing device which can eliminate a troublesome proo 
ess for unifying formats and allow a high-quality editing 
work where degradation of video data following recom- 
pression is prevented, even when mixed video source 15 
data in many kinds of compression formats exists. 

Reference numerals 



software decoder realized by a software. 

The nonlinear editing device according to claims 2 
or 3, wherein said first multi-format decoder and 
said second mufti-format decoder are sequentially 
switched with each other responsive to a transition 
point of compression formats when compression 
formats of video data read out from said storage are 
various. 

The nonlinear editing device according to claims 2 
or 3, wherein said second mutttforrnat decoder 
decompresses video data in a part at least one of 
just before a transition point of compression for^ 
mats and just after of the transition point, and said 
first multi-format decoder decompresses video data 
in the other part, when compression formats of the 
video data read out from said storage ana various. 



[0073] 
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6. The nonlinear editing device according to claims 2 






or 3 further comprising 


101 


HARD DISK UNIT 






102 


FIRST MULTUFORMAT DECODER 




a digital video effector for synthesizing output 


103 


FIRST FORMAT CONVERTER 




data of said first multi-format decoder and out- 


104 


SECOND MULTI-FORMAT DECODER 
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put data of said second multi-format decoder. 


106 


SECOND FORMAT CONVERTER 






106 


DIGITAL VIDEO EFFECTOR 




7- The nonlinear editing device according to claim 4 


107 


THIRD FORMAT CONVERTER 




further comprising 


108 


VIDEO OUTPUT SECTION 






109 


EDITING CONTROL SECTION 


30 


a digital video effector for synthesizing' output 
data of said first multi-format decoder and out- 


Claims 




put data of said second multi-format decoder. 



1. A nonlinear editing device for editing video data, 
audio data, or video data and audio data, said non- as 
Unear editing device comprising: 

a storage for recording video data compressed 
and encoded in a plurality of Kinds of compres- 
sion formats keeping the compression formats; 40 
and 

a first multi-format decoder for decompressing 
the Video data recorded in the storage in at 
[eaet two or more compression formats of the 
plurality of kinds of compression formats. 45 

2. The nonlinear editing device according to claim 1 
further comprising 

a second multi-format decoder for decom- so 
pressing the video data recorded in said stor- 
age in at least one or more compression 
formats of the plurality of kinds of compression 
formats. 

$5 

3. The nonlinear editing device according to claim 2 T 
wherein at least one of said first multi-format 
decoder and said second multi-format decoder Is a 



8. The nonlinear editing device according to claim 5 
further comprising 

a digital video effector for synthesizing output 
data of said first multi-format decoder and out- 
put data of said second multi-format decoder. 

9- The nonlinear editing device according to claims 1 , 
2, or 3 further comprising 

a first format converter for converting output 
data of said first multi-format decoder with at 
least one of SD (Standard Definition) / HD 
(High Definition) conversion, HD/SD conver- 
sion, NTSC (National Television System Com- 
mittee) / PAL (Phase Alternation Line) 
conversion, and PAUNTSC conversion. 

10. The nonlinear editing device according to claim 4 
further comprising 

a first format converter for converting output 
data of said first multi-format decoder with at 
least one of SD/HD conversion, HD/SD conver- 
sion, NTSC/PAL conversion, and PAL/NTSC 
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11. The nonlinear editing device according to claim 5 
further comprising 

a first formal converter for converting output 
data of said first mum-format decoder with at 
least one of SD/HD conversion, HD/SD conver- 
sion, NTSC/PAL conversion, and PAUNTSC 
conversion. 

12. The nonlinear editing device according to claim 6 
further comprising 

a first format converter for converting the out- 
put data of said first multMormat decoder with 
at least one of SD/HD conversion, HD/SD con- 
version, NT&C/PAL conversion, and 
PAL/NTSC conversion. 

13. The nonlinear editing device according to claims 7 
or 8 further comprising 

a first format converter for converting, the out- 
put data of said first multi-format decoder with 
at least one of SD/HD conversion, HD/SD con- 
version, NTSC/PAL conversion, and 
PAL/NTSC conversion. 

14. The nonlinear editing device according to claims 2 
cr 3 further comprising 

a second format converter for converting out- 
put data of said second multi-format decoder 
with at least one of SD/HD conversion, HD/SD 
conversion, NTSC/PAL conversion, and 
PAL/ NTS C conversion. 



is. The nonlinear editing device according to claim 4 
further comprising 

a second format converter, for converting out- 
put data of said second multi -format decoder 
with at least one of SD/HD conversion, HD/SD 
conversion, NTSC/PAL conversion, and 
PAL/NTSC conversion. 

16. The nonlinear editing device according to claim 5 
further comprising 

a second format converter for converting out- 
put data of said second multi-format decoder 
with at least one of SD/HD conversion, HD/SD 
conversion, NTSC/PAL conversion, and 
PAUNTSC conversion. 

17, The nonlinear editing device according to claim 6 
further comprising 



a second format converter for converting the 
output data of said second multi-format 
decoder with at least one of SD/HD conversion, 
HD/SD conversion, NTSC/PAL conversion, and 
PAL/NTSC conversion. 

18. The nonlinear editing device according to claims 7 
or 8 further comprising 

a second format converter for converting the 
output data of said second multi-format 
decoder with at least one of SD/HD conversion, 
HD/SD conversion. NTSC/PAL conversion, and 
PAL/NTSC conversion. 

1 3. The nonlinear editing device according to claim 9 
further comprising 

a second format converter for converting out- 
put data of said second multi-format decoder 
with at least one of SD/HO conversion, HD/SD 
conversion, NTSC/PAL conversion, and 
PAL/NTSC conversion. 



25 20. The nonlinear editing device according to claims 1 0 
or 1 1 further comprising 



a second format converter for converting out- 
put data of said second multi-format decoder 
with at least one of SD/HD conversion, HD/SD 
conversion, NTSC/PAL conversion, and 
PAL/NTSC conversion. 
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21- The nonlinear editing device according to claim 12 
further comprising 

a second format converter for converting the 
output data of said second multi-format 
decoder with at least one of SD/HD conversion, 
HD/SD conversion. NTSC/PAL conversion, and 
PAL/NTSC conversion. 

• 22. The nonlinear editing device according to claim 13 
further comprising 

a second format converter for converting the 
output data of said second multi-format 
decoder with at least one of SD/HD conversion, 
HD/SD conversion, NTSC/PAL conversion, and 
PAUNTSC conversion. 

23. The nonlinear editing device according to claim 6 
further comprising 

a third format converter for converting output 
data of said digital video effector with at (east 
one of SD/HD conversion, HD/SD conversion, 
NTSC/PAL conversion, and PAL/NTSC eonver- 
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24. The nonlinear editing device according to claims 7 
or 8 further comprising 

a third format converter for converting output 
data of said digital video effector with at least 
one of SD/HD conversion, HD/SD conversion, 
NTSC/PAL conversion, and PAL7NTSC conver- 
sion - 

25. The nonlinear editing device according to claim 12 
further comprising 

a third format converter for converting output 
data of said digital video effector with at least 
one of the SD/HD conversion, the HD/SD con- 
version, the NTSC/PAL conversion, and the 
PAL/NTSC conversion. 

26. The nonlinear editing device according to claim 13 
further comprising 

a third format converter for converting output 
data of said digital video effector with at least 
one of the SD/HD conversion, the HD/SD con- 
version, the NTSC/PAL conversion, and the 
PAL/NTSC conversion. 

27. The nonlinear editing device according to claim 1 7 
further comprising 

a third format converter for converting output 
data of said digital video effector with at least 
one of the SD/HD conversion, the HD/SD con- 
version, the NTSC/PAL conversion, and the 
PAL/NTSC conversion. 

2fi. The nonlinear editing device according to claim 1 8 
further comprising 

a third format converter for converting output 
data Of said digital video effector with at leaet 
on* of the SD/HD conversion, the HD/SD con- 
version, the NTSC/PAL conversion, and th9 
PAL/NTSC conversion. 

29. The nonlinear editing device according to claim 21 
further comprising 

a third format converter for converting output 
data cf said digital video effector with at least 
one of the SD/HD conversion, the HD/SD con- 
version, the NTSC/PAL conversion, and the 
PAUNTSC conversion, 

30. The nonlinear editing device according to claim 22 
further comprising 
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a third format converter for converting output 
data of said digital video effector with ai least 
one of the SD/HD conversion, the HD/SD con- 
version, the NTSC/PAL conversion, and the 
PAL/NTSC conversion. 

31. A nonlinear editing method for editing vvideo data, 
audio data, or video data and audio data, said non- 
linear editing method comprising 

a first decoding step tor sequentially decom- 
pressing video data which is recorded tn a stor- 
age that can record the video data compressed 
and encoded in a plurality of kinds of compres- 
sion formats keeping the compression formats 
. and has at least two or more compression for- 
mats of the plurality of kinds of compression 
formats. 

32. The nonlinear editing method according to claim 31 
further comprising 

a second decoding step for sequentially 
decompressing video data recorded in the stor- 
age in at (east one or more compression for- 
mats of the plurality Of Wnds of compression 
formats. 

33. The nonlinear editing method according to claim 
32, wherein said first decoding step and said sec- 
ond decoding step are sequentially switched with 
each other responsive to a transition point of com- 
pression formats when compression formats of the 
video data read out from the storage are various. 
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34. The nonlinear editing method according to claim 
32, wherein video data in a part at least one ot just 
oetore a transition point ot compression formats 
and just after of the point are decompressed in said 
second decoding step, and video data in the other 
part is decompressed in said first decoding step, 
when compression formats of the video data read 
out from said storage are various. 

35. The nonlinear editing method according to claims 
32, 33, or 34 further comprising 

a video effect step for synthesizing output data 
obtained in said first decoding step and output 
data obtained in said second decoding step. 

36. The nonlinear editing method according to claim 31 
further comprising 

a first format converting step for converting out- 
put data of said first decoding step with at least 
one of SD/HD conversion, HD/SD conversion, 
NTSC/RAL conversion, and PAL/NTSC conver- 
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37. The nonlinear editing method according to claims 
32, 33, or 34 further comprising 

* first format converting step for converting out- 
put data obtained in said first decoding step 
with at (east one of SD/HD conversion, HD/SD 
conversion, NTSC/PAL conversion, and 
PAL/NTSC conversion. 

38. The nonlinear editing method according to claim 3S 
further comprising 

e first format converting step for converting out- 
put data obtained m said first decoding step 
with at least one of SD/HD conversion, HD/SD 
conversion, NTSC/PAL conversion, and 
PALVNTSC conversion. 

39. The nonlinear editing method according to claims 
32, 33, or 34 further comprising 

a second format converting step for converting 
output data obtained in said second decoding 
step with at least one of SD/HD conversion, 
HD/SD conversion, NTSC/PAL conversion, and 
PAL/NTSC conversion. 

40. The nonlinear editing method according to claim 35 
further comprising 

a second format converting step for converting 
the output data obtained in said second decod- 
ing step w'rth at least one of SD/HD conversion, 
HD/SD conversion, NTSC/PAL conversion, and 
PAL/NTSC conversion. 

41. The nonlinear editing method according to claim 37 
further comprising 

a second format converting step for converting 
the output data obtained In said second decod- 
ing step with at least one of SD/HD conversion, 

HD/SD conversion, NTSC/PAL conversion, and 

PAL/NTSC conversion. 

42. The nonlinear editing method according to claim 38 
further comprising 

a second format converting step for converting 
the output data obtained in said second decod- 
ing step with at ieast one of SD/HD conversion, 
HD/SD conversion, NTSC/PAL conversion, and 
PAL/NTSC conversion. 

43. The nonlinear editing method according to claim 35 
further comprising 



a third format converting step for converting 
output data obtained in said video effect step 
with at least one of SD/HD conversion, HD/SD 
conversion, NTSC/PAL conversion, and 
PAL/NTSC conversion. 

44- The nonlinear editing method according to claim 36 
further comprising 

a third format converting step for converting 
output data obtainea In said video effect step 
with at least one of the SD/HD conversion, the 
HD/SD conversion, the NTSC/PAL conversion, 
and the PAUWTSC conversion. 

45; The nonlinear editing method according to claims 
40 or 42 further comprising 

a third format converting step for converting 
output data obtained in said video effect step 
with at least one of the SD/HD conversion, the 
HD/SD conversion, the NTSC/PAL conversion, 
and the PAL/NTSC conversion. 
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